Perceptions of Alignment and Personality in Generated Dialogue

Variation in language style can lead to different perceptions of the interaction, and different behaviour outcomes. Using the CRAG 2 language generation system we examine how accurately judges can perceive character personality from short, automatically generated dialogues, and how alignment (similarity between speakers) alters judge perceptions of the characters’ relationship. Whilst personality perception of our dialogues is consistent with perceptions of human behaviour, we find that the introduction of alignment leads to negative perceptions of the dialogues and the interlocutors’ relationship. A follow up evaluation study of the perceptions of different forms of alignment in the dialogues reveals that while similarity at polarity, topic and construction levels is viewed positively, similarity at the word level is regarded negatively. We discuss our findings in relation to the literature and in the context of dialogue systems.